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FURTHER RESULTS ON THE H-TEST OF DURBIN FOR 
STABLE AUTOREGRESSIVE PROCESSES 



FREDERIC PROIA 

Abstract. The purpose of this paper is to investigate the asymptotic behav- 
ior of the Durbin- Watson statistic for the stable p— order autoregressive process 
when the driven noise is given by a first-order autoregressive process. It is an 
extension of the previous work of Bercu and Proia devoted to the particular case 
p = 1. We establish the almost sure convergence and the asymptotic normality for 
both the least squares estimator of the unknown vector parameter of the autore- 
gressive process as well as for the serial correlation estimator associated with the 
driven noise. In addition, the almost sure rates of convergence of our estimates are 
also provided. Then, we prove the almost sure convergence and the asymptotic 
normality for the Durbin- Watson statistic and we derive a two-sided statistical 
*vj ■ procedure for testing the presence of a significant first-order residual autocorre- 

lation that appears to clarify and to improve the well-known h-test suggested by 
Durbin. Finally, we briefly summarize our observations on simulated samples. 

43 ' 1. INTRODUCTION 

The Durbin- Watson statistic was originally introduced by the eponymous econo- 
metricians Durbin and Watson [16] , [17] , [18] in the middle of last century, in order 
to detect the presence of a significant first-order autocorrelation in the residuals 
from a regression analysis. The statistical test worked pretty well in the indepen- 
dent framework of linear regression models, as it was specifically investigated by 
Tillman [32]. While the Durbin- Watson statistic started to become well-known in 
Econometrics by being commonly used in the case of linear regression models con- 
^ | taining lagged dependent random variables, Malinvaud [28] and Nerlove and Wallis 

[30] observed that its widespread use in inappropriate situations were leading to in- 
adequate conclusions. More precisely, they noticed that the Durbin- Watson statistic 
was asymptotically biased in the dependent framework. To remedy this misuse, al- 
ternative compromises were suggested. In particular, Durbin [TJ] proposed a set of 
revisions of the original test, as the so-called t-test and h-test, and explained how 
to use them focusing on the first-order autoregressive process. It inspired a lot of 
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works afterwards. More precisely, Maddala and Rao [27] . Park [3T] and then Inder 
[23j . [21] and Durbin [T5] looked into the approximation of the critical values and 
distributions under the null hypothesis, and showed by simulations that alternative 
tests significantly outperformed the inappropriate one, even on small-sized samples. 
Additional improvements were brought by King and Wu [25] and lately, Stocker [32] 
gave substantial contributions to the study of the asymptotic bias resulting from 
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2 FREDERIC PROIA 

the presence of lagged dependent random variables. In most cases, the first-order 
autoregressive process was used as a reference for related research. This is the reason 
why the recent work of Bercu and Proia jl] was focused on such a process in order 
to give a new light on the distribution of the Durbin- Watson statistic under the 
null hypothesis as well as under the alternative hypothesis. They provided a sharp 
theoretical analysis rather than Monte-Carlo approximations, and they proposed a 
statistical procedure derived from the Durbin- Watson statistic. They showed how, 
from a theoretical and a practical point of view, this procedure outperforms the 
commonly used Box-Pierce [7J and Ljung-Box [6] statistical tests, in the restrictive 
case of the first-order autoregressive process, even on small-sized samples. They 
also explained that such a procedure is asymptotically equivalent to the h-test of 
Durbin [H] for testing the significance of the first-order serial correlation. This work 
[I] had the ambition to bring the Durbin- Watson statistic back into light. It also 
inspired Bitseki Penda, Djellout and Proia [5] who established moderate deviation 
principles on the least squares estimators and the Durbin- Watson statistic for the 
first-order autoregressive process where the driven noise is also given by a first-order 
autoregressive process. 

Our goal is to extend of the previous results of Bercu and Proia [I] to p— order 
autoregressive processes, contributing moreover to the investigation on several open 
questions left unanswered during four decades on the Durbin- Watson statistic [2], 
[T5] . [30] . One will observe that the multivariate framework is much more difficult 
to handle than the scalar case of [I]. We will focus our attention on the p— order 
autoregressive process given, for all n > 1, by 

X n = 9\X n _i + . . . + 6 p X n _ p + e n 

where the unknown parameter 6 — \9\ 9 2 ... # p ) is a nonzero vector such that 
||# ||i < 1, and the unknown parameter |p| < 1. Via an extensive use of the theory of 
martingales [12], [21], we shall provide a sharp and rigorous analysis on the asymp- 
totic behavior of the least squares estimators of 9 and p. The previous results of 
convergence were first established in probability [28], [30], and more recently almost 
surely [I] in the particular case where p = 1. We shall prove the almost sure conver- 
gence as well as the asymptotic normality of the least squares estimators of 9 and 
p in the more general multivariate framework, together with the almost sure rates 
of convergence of our estimates. We will deduce the almost sure convergence and 
the asymptotic normality for the Durbin- Watson statistic. Therefore, we shall be in 
the position to propose further results on the well-known h-test of Durbin [H] for 
testing the significance of the first-order serial correlation in the residuals. We will 
also explain why, on the basis of the empirical power, this test procedure outper- 
forms Ljung-Box [6] and Box-Pierce [7J portmanteau tests for stable autoregressive 
processes. We will finally show by simulation that it is equally powerful than the 
Breusch-Godfrey [8], [19] test and the h-test [14] on large samples, and better than 
all of them on small samples. 
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The paper is organized as follows. Section 2 is devoted to the estimation of the 
autoregressive parameter. We establish the almost sure convergence of the least 
squares vector estimator of 9 to the limiting value 

(1.2) 9*=a(I p -9 pP J p )f3 

where I p is the identity matrix of order p, J p is the exchange matrix of order p, and 
where a and /3 will be calculated explicitly. The asymptotic normality as well as the 
quadratic strong law and a set of results derived from the law of iterated logarithm 
are provided. Section 3 deals with the estimation of the serial correlation parameter. 
The almost sure convergence of the least squares estimator of p to 

(i.3) P * = e pP e; 

where 9* stands for the p— th component of 9* is also established along with the 
quadratic strong law, the law of iterated logarithm and the asymptotic normality. 
It enables us to establish in Section 4 the almost sure convergence of the Durbin- 
Watson statistic to 
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(1.4) u=-i\v-p\ 

together with its asymptotic normality. Our sharp analysis on the asymptotic be- 
havior of the Durbin- Watson statistic remains true whatever the values of the pa- 
rameters 9 and p as soon as ||0||i < 1 and \p\ < 1, assumptions resulting from the 
stability of the model. Consequently, we are able in Section 4 to propose a two-sided 
statistical test for the presence of a significant first-order residual autocorrelation 
closely related to the h-test of Durbin [T4]. A theoretical comparison as well as a 
sharp analysis of both approaches are also provided. In Section 5, we give a short 
conclusion where we briefly summarize our observations on simulated samples. We 
compare the empirical power of this test procedure with the commonly used port- 
manteau tests of Box-Pierce [7] and Ljung-Box [6], with the Breusch-Godfrey test 
[S], [12] and the h-test of Durbin [13] . Finally, the proofs related to linear algebra 
calculations are postponed in Appendix A and all the technical proofs of Sections 
2 and 3 are postponed in Appendices B and C, respectively. Moreover, Appendix 
D is devoted to the asymptotic equivalence between the h-test of Durbin and our 
statistical test procedure. 

Remark 1.1. In the whole paper, for any matrix M, M' is the transpose of M. For 
any square matrix M, tr(M) ; det(M) ; |||M|||i and p(M) are the trace, the determi- 
nant, the 1-norm and the spectral radius of M, respectively. In addition, \ min (M) 
and X ma x{M) denote the smallest and the largest eigenvalues of M, respectively. For 
any vector v, \\v\\ stands for the euclidean norm of v and \\v\\i is the 1-norm of v. 

Remark 1.2. Before starting, we denote by I p be the identity matrix of order p, J p 
the exchange matrix of order p and e the p— dimensional vector given by 

(1 0\ /0 \\ (\\ 
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2. ON THE AUTOREGRESSIVE PARAMETER 

Consider the p— order autoregressive process given by (II .11 where we shall sup- 
pose, to make calculations lighter without loss of generality, that the square-integrable 
initial values X = e and X_i, X_ 2 , . . . , X_ p = 0. In all the sequel, we assume that 
(V n ) is a sequence of square-integrable, independent and identically distributed ran- 
dom variables with zero mean and variance a 2 > 0. Let us start by introducing 
some notations. Let $^ stand for the lag vector of order p, given for all n > 0, by 

(2.1) K={ X n Xn-1 ••• X n _ p+1 )'. 

Denote by S n the positive definite matrix defined, for all n > 0, as 



(2-2) S n = J2®K + S 



k=0 



where the symmetric and positive definite matrix S is added in order to avoid an 
useless invertibility assumption. For the estimation of the unknown parameter 9, it 
is natural to make use of the least squares estimator which minimizes 

n 

V n (6) = J2(X k -6'n-i) 2 - 
k=i 

A standard calculation leads, for all n > 1, to 

(2.3) e n = (s nr . 1 r i ^2n~iXk. 



fe=i 



Our first result is related to the almost sure convergence of 9 n to the limiting value 
9* = a (I p — 9 p pJp) (3, where 

(2.4) a 



[l-9 p p)(l + 9pp) 



(2.5) f3=(9i+ P 9 2 -9 lP ... 9 p -9 p - lP )'. 
Theorem 2.1. We have the almost sure convergence 

(2.6) lim 9 n = 9* a.s. 



n—yoo 



Remark 2.1. In the particular case where p = 0, we obtain the strong consistency 
of the least squares estimate in a stable autoregressive model, already proved e.g. in 
\, under the condition of stability \\0\\i < 1. 



TESTING RESIDUALS FROM A STABLE AUTOREGRESSIVE PROCESS 



Let us now introduce the square matrix B of order p + 2, partially made of the 
elements of given by (12. 5ft . 

/ 1 -fil -fa -Pp-1 

-01 1-/32 -03 ~Pp 

-02 -01-03 1-04 %P 



(2.7) B 



-0 P 

e pP 





o p p\ 







-0p 
V °pp 



-0p-l + %p 

-0 P 



-0p-2 

-0P-1 



-01 

-02 



-01 





1 



/ 



Under our stability conditions, we are able to establish the invertibility of B in 
Lemma 12.11 The corollary that follows will be useful in the next section. 



Lemma 2.1. Under the stability conditions ||0||i < 1 and \p\ < 1, the matrix B 
given by (12. 7p is invertible. 

Corollary 2.1. By virtue of Lemma \2.1\ the submatrix C obtained by removing 
from B its first row and first column is invertible. 

From now on, A G M. p+2 is the unique solution of the linear system BA = e, i.e. 

(2.8) A = B~ l e 

where the vector e has already been defined in Remark 1 1.11 but in higher dimension. 
Denote by Ao, . . . , A p+ i the elements of A and let A p be the Toeplitz matrix of order 
p associated with the first p elements of A, that is 

/ Ao Ai A2 
Ai Ao Ai 

(2.9) A, 



Ap_i\ 



A 



V -2 



\Ap_i A p _2 Ap_ 3 Ao / 

Via the same lines, we are able to establish the invertibility of A p in Lemma | 

Lemma 2.2. Under the stability conditions \\0\\i < 1 and \p\ < 1, for all p > 1, the 
matrix A p given by (12. 9p is positive definite. 

In light of foregoing, our next result deals with the asymptotic normality of 6 n . 

Theorem 2.2. Assume that (V n ) has a finite moment of order 4- Then, we have 
the asymptotic normality 

(2.10) v^(3n-0*) ^A/-(0,£ e ) 
where the asymptotic covariance matrix is given by 

(2.11) S e = a 2 (I p - 6 pP J p ) A; 1 (I p - 6 pP J p ) . 

Remark 2.2. The covariance matrix Eg is invertible under the stability conditions. 
Furthermore, due to the way it is constructed, Tig is bisymmetric. 
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Remark 2.3. In the particular case where p = 0, Eg reduces to A" 1 . This is a 
well-known result related to the asymptotic normality of the Yule- Walker estimator 
for the causal autoregressive process that can be found e.g. in Theorem 8.1.1 of [9]. 

After establishing the almost sure convergence of the estimator 9 n and its asymptotic 
normality, we focus our attention on the almost sure rates of convergence. 

Theorem 2.3. Assume that (V n ) has a finite moment of order 4- Then, we have 
the quadratic strong law 

(2.12) lim V (e k - 9*) (0 k - 9*) = E, a.s. 

n->oo log n z — ' V / V / 

to fc=l 

where T,g is given by (12. lip . In addition, for all v e W, we also have the law of 
iterated logarithm 

/ \l/2 / xl/2 

limsup ( -—4 J v' (e n -9*) = - liminf ( — — ^ J v' (e n - 9 A 

n^oo \21oglogn/ V / n^oo \21oglogn/ V 

(2.13) = ^v'Y> e v a.s. 
Consequently, 

(2.14) limsup (-—^ ) (e n - 9*) (e n - 6*\ = E e a.s. 

n^oo V21oglogn/ V / V / 

In particular, 

777-n \\ 9 n- e *\\ = tr ( S e) a.s. 

21oglogn/ " " 

Remark 2.4. It clearly follows from (12.1 2p that 

1 " 

(2.16) lim V 1 1 9 k - 9* 1 1 2 = tr (E fl ) a.s. 

to fe=i 
Furthermore, from (12.151) . we have the almost sure rate of convergence 

(2.17) \\9 n — 9 \\ = O [ a.s. 

V n J 

Proof. The proofs of Lemma 12.11 and Lemma 12.21 are given in Appendix A while 
those of Theorems 12.11 to 12.31 may be found in Appendix B. □ 

To conclude this section, let us draw a parallel between the results of j4] and the 
latter results for p = 1. In this particular case, /3 and a reduce to {9 + p) and 
(1 — 9p)~ 1 (l + 9p)~ x respectively, and it is not hard to see that we obtain the almost 
sure convergence of our estimate to 



TESTING RESIDUALS FROM A STABLE AUTOREGRESSIVE PROCESS 



In addition, a straightforward calculation leads to 

^ (i-fl 2 )(i-flp)(i- P 2 ; 



One can verify that these results correspond to Theorem 2.1 and Theorem 2.2 of [3]. 

3. ON THE SERIAL CORRELATION PARAMETER 

This section is devoted to the estimation of the serial correlation parameter p. 
First of all, it is necessary to evaluate, at stage n, the residual set (s n ) resulting 
from the biased estimation of 9. For all 1 < k < n, let 

(3-1) e k = X k -e^ k _ v 

The initial value £q ma y b e arbitrarily chosen and we take Eq = X Q for a matter of 
simplification. Then, a natural way to estimate p is to make use of the least squares 
estimator which minimizes 

n 

V«(p) = J2(e k -pe k - 1 ) 2 . 

k=l 

Hence, it clearly follows that, for all n > 1, 



- 1 



(3-2) Pn=[J2^-l) J2^ k - X - 

\k=l J fc=l 

It is important to note that one deals here with a scalar problem, in contrast to the 
study of the estimator of 9 in Section 2. Our goal is to obtain the same asymptotic 
properties for the estimator of p as those obtained for each component of the one of 
6. However, one shall realize that the results of this section are much more tricky 
to establish than those of the previous one. 

We first state the almost sure convergence of p n to the limiting value p* = p p9*. 

Theorem 3.1. We have the almost sure convergence 

(3.3) lim p n = p* a.s. 

n— >oo 

Our next result deals with the joint asymptotic normality of 9 n and p n . For that 
purpose, it is necessary to introduce some additional notations. Denote by P the 
square matrix of order p + 1 given by 

Pb 



where 

P B = a^p-OppJpjAp 1 , 

Pl = J P {l P -e pP J p ){a6 p pA~ 1 e + e;i3), 
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Furthermore, let us introduce the Toeplitz matrix A p+1 of order p + 1 which is the 
extension of A p given by ( 12. 9 p to the next dimension, 

(3 ' 5) Ap+1= \K'J P V 

with A^ = (Ai A2 ... A p ) , and the positive semi definite covariance matrix Y of 
order p + 1, given by 

(3.6) T = PA P+1 P'. 

Theorem 3.2. Assume that (V n ) has a finite moment of order 4- Then, we have 
the joint asymptotic normality 

(3.7) Vn-(i n Z 9 l) ^A/-(0,r). 
In particular, 

(3.8) Vn~(p n -P*) ^A/-(0,a p 2 ) 
where a 2 = r p+ i iP+ i is the last diagonal element ofT. 

Remark 3.1. The covariance matrix V has the following explicit expression, 

Eg 9 p p JpT.ee 

pe'TeJp a 2 

where 

(3.9) a 2 p = PlA p P L -2a- 1 9;A l p '.J p P L +(a- l 9;) 2 ^- 

Remark 3.2. The covariance matrix T is invertible under the stability conditions 
if and only if 9* 7^ since, by a straightforward calculation, 

det(r) = a 2 ^ (9;) 2 det(A p+1 ) (^7^)' 

according to Lemma \2.2\ and noticing that (I p — 9 p pJ p ) is strictly diagonally domi- 
nant, thus invertible. As a result, the joint asymptotic normality given by (13. 7p is 
degenerate in any situation such that 9* = 0, that is 

(3.10) 9 p -9 p _ lP = 9 pP (9 1 + p). 

Moreover, (13.81) holds on {9 P — 9 p _ x p 7^ 9 p p{9\ + p)} U {9 P 7^ 0,p 7^ 0}, otherwise the 
asymptotic normality associated with ~p n is degenerate. In fact, a more restrictive 
condition ensuring that (13.81) still holds may be {9 P 7^ 0}, i.e. that one deals at least 
with a p— order autoregressive process. This restriction seems natural in the context 
of the study and can be compared to the assumption {9 7^ 0} in|. Theorem 3.2 of 
[I] ensures that the joint asymptotic normality is degenerate under {9 = —p}. One 
can note that such an assumption is equivalent to (13.101) in the case of the p— order 
process, since both of them mean that the last component of 9* has to be nonzero. 

The almost sure rates of convergence for p n are as follows. 



2 
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Theorem 3.3. Assume that (V n ) has a finite moment of order 4- Then, we have 
the quadratic strong law 



(3.H) lim- Ysi^-P* 

rwoo log n *■ — 4 \ 
6 fc=i 



2 

= a 2 p a.s. 



where a is given by (13. 9p . In addition, we also have the law of iterated logarithm 

limsup ( — — I (p n -p*) = - liminf ( — — ) (p n - p* ) , 

n^oo \2logiognJ V / n->oo \2loglognJ V / 

(3.12) = o p a.s. 
Consequently, 

(3.13) limsup I — — ) (p n - p* ) =a 2 a.s. 

Remark 3.3. It clearly follows from ( 13. 13|) that we have the almost sure rate of 
convergence 

(3.14) \Pn-p) =0 1 1 a.s. 

As before, let us also draw the parallel between the results of [1] and the latter results 
for p = 1. In this particular case, we immediately obtain p* = 9p9* . Moreover, an 
additionnal step of calculation shows that 

l-( 



a 2 



{(6 + p) 2 (l + 6p) 2 + (6p) 2 (l-6 2 )(l- P 2 )) 



p (l + #p) 3 

One can verify that these results correspond to Theorem 3.1 and Theorem 3.2 of jl]. 
Besides, the estimators of 9 and p are self-normalized. Consequently, the asymptotic 
variances Eg and a 2 do not depend on the variance a 2 associated with the driven 
noise (V^). To be complete and provide an important statistical aspect, it seemed 
advisable to suggest an estimator of the true variance a 2 of the model, based on 
these previous estimates. Consider, for all n > 1, the estimator given by 



e k 2 



(3-15) ^(i-^fc^E 

fc=0 

where 9 Ptn stands for the p— th component of 9 n . 
Theorem 3.4. We have the almost sure convergence 
(3.16) lim a 2 = a 2 a.s. 

n— >oo 

Proof. The proofs of Theorems 13.11 to 13.31 are given in Appendix C. The one of 
Theorem 13.41 is left to the reader as it directly follows from that of Theorem 13.11 □ 
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4. ON THE DURBIN- WATSON STATISTIC 

We shall now investigate the asymptotic behavior of the Durbin- Watson statistic 
for the general autoregressive process [16], [17], [18], given, for all n > 1, by 

fc=0 / fe=l 

As mentioned, the almost sure convergence and the asymptotic normality of the 
Durbin- Watson statistic have previously been investigated in [3] in the particular 
case where p — 1. It has enabled the authors to propose a two-sided statistical test 
for the presence of a significant residual autocorrelation. They also explained how 
this statistical procedure outperformed the commonly used Ljung-Box [BJ and Box- 
Pierce [7J portmanteau tests for white noise in the case of the first-order autoregres- 
sive process, and how it was asymptotically equivalent to the h-test of Durbin [13], 
on a theoretical basis and on simulated data. They went even deeper in the study, 
establishing the distribution of the statistic under the null hypothesis "p = po", 
with |p | < 1, as well as under the alternative hypothesis "p ^ p ", and noticing the 
existence of a critical situation in the case where 9 = —p. This pathological case 
arises when the covariance matrix T given by (13. 6 p is singular, and can be compared 
in the multivariate framework to the content of Remark 13.21 Our goal is to obtain 
the same asymptotic results for all p > 1 so as to build a new statistical procedure 
for testing serial correlation in the residuals. In this paper, we shall only focus our 
attention on the test "p = 0" against "p ^ 0", of increased statistical interest. We 
shall see below that from a theoretical and a practical point of view, our statistical 
test procedure clarifies ans outperforms the h-test of Durbin. In particular, it avoids 
the presence of an abstract variance estimation likely to generate perturbations on 
small-sized samples. In the next section, we will observe on simulated data that the 
procedure proposed in Theorem 14.41 is more powerful than the portmanteau tests [B], 
[7] , often used for testing the significance of the first-order serial correlation of the 
driven noise in a p— order autoregressive process. 

First, one can observe that D n and p n are asymptotically linked together by an 
affine transformation. Consequently, the asymptotic behavior of the Durbin- Watson 
statistic directly follows from the previous section. We start with the almost sure 
convergence to the limiting value D* = 2(1 — p*). 

Theorem 4.1. We have the almost sure convergence 

(4.2) lim D n = D* a.s. 

Our next result deals with the asymptotic normality of D n . It will be the keystone 
of the statistical procedure deciding whether residuals have a significant first-order 
correlation or not, for a given significance level. Denote 

(4.3) a% = Aa 2 p 
where the variance a 2 is given by (13. 9p . 
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Theorem 4.2. Assume that (V n ) has a finite moment of order 4- Then, we have 
the asymptotic normality 

(4.4) y/E(D n -D^ -^Af(0,a 2 D ). 
Remark 4.1. We immediately deduce from ( 14.41) that 

(4.5) ^(4-Df^x 2 

where x 2 has a Chi-square distribution with one degree of freedom. 

Let us focus now on the almost sure rates of convergence of D„. 

Theorem 4.3. Assume that (V n ) has a finite moment of order 4- Then, we have 
the quadratic strong law 



o D a.s. 



(4.6) lim-LyVBfc-D' 

n->oo log U z — ' V 
to fc=l 

where a 2 D is given by (14.31) . In addition, we also have the law of iterated logarithm 



,1/2 / \ 1/2 



limsup — -: [D n -D*\ = -liminf — [D n -D' 

n _>oo \2l0gl0gny V / n-^oo \21oglOgn 

(4.7) = o~d a.s. 

Consequently, 



> 



n 



2 



(4.8) limsup ( — — \ I ( D n - D* ) = a 2 a.s. 

V ; n^J \2\og\ogn ) V J D 

Remark 4.2. It clearly follows from (14. 8 j) that we have the almost sure rate of 
convergence 

(4.9) (B„- D .) 2 = (!2i^ij a ., 

We are now in the position to propose the two-sided statistical test built on the 
Durbin- Watson statistic. First of all, we shall not investigate the particular case 
where 9 p = since our procedure is of interest only for autoregressive processes of 
order p. One wishes to test the presence of a significant serial correlation, setting 

H : "p = 0" against Hi : "p^O". 

Theorem 4.4. Assume that (V n ) has a finite moment of order 4, % ^ and 9* ^ 0. 
Then, under the null hypothesis Ho '■ "p = 0" , 



n (^ „V c. 2 



(4.10) - (Dn - 2 f^ x 
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where 9 Ptn stands for the p—th component of 9 n , and where \ 2 has a Chi-square 
distribution with one degree of freedom. In addition, under the alternative hypothesis 
fti : WO", 



■/?, 



2 



(4.11) lim^^LD n -2 = +oo a.s. 

n-xx 4/9 2 V / 

From a practical point of view, for a significance level a where < a < 1, the 
acceptance and rejection regions are given by A = [0, z a ] and 1Z = ]z a , +oo[ where 
z a stands for the (1 — a)— quantile of the Chi-square distribution with one degree of 
freedom. The null hypothesis Ho wm n °t be rejected if the empirical value 

n (~ \ 2 

D n -2) < z„, 



40* V '" 

p,n 

and will be rejected otherwise. 

Remark 4.3. In the particular case where 9* = 0, the test statistic do not respond 
under H± as described above. To avoid such situation, we suggest to make use 
of Theorem \2.S\ for testing beforehand whether 9 P)n is significantly far from zero. 
Besides, testing Ho : "p = 0" with 9* = amounts to testing the significance of the 
p—th coefficient of the model, not rejected under {9 P ^ 0}. Roughly speaking, under 
{9 P y^ 0} fl {9* = 0}, we obviously have p^ and the use of Theorem 4-4 would be 
irrelevant since H,\ is certainly true. 

As previously mentioned, the statistical procedure of Theorem 14.41 appears to be a 
substantial clarification of the h-test of Durbin p3]. To be more precise, formula 
(12) of [H] suggests to make use of the test statistic 
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(4.12) Hi 




-nV n (0i, n ) 

where V n (#i >n ) is the least squares estimate of the variance of the first element of 9 n , 
and to test it as a standard normal deviate. The presence of an abstract variance 
estimator not only makes the procedure quite tricky to interpret, but also adds some 
vulnerability on small-sized samples, as will be observed in the next section. The 
almost sure equivalence between both test statistics is shown in Appendix D. 

Remark 4.4. The h-test of Durbin [E] is based on the normality assumption on the 
driven noise (V n ). As a consequence, (X n ) is a Gaussian process and the maximum 
likelihood strategy is suitable not only to provide the estimates, but also to determine 
their conditional distributions. One can observe that all our results hold without 



any Gaussianity assumption on (V n ). Hence, Theorem \4-4\ appears to generalize the 
h-test of Durbin. 

Proof. The proofs of Theorems 14.11 to 14.31 are left to the reader as they follow es- 
sentially the same lines as those given in Appendix C of jl] . Theorem 14.41 is an 
immediate consequence of Theorem I4.2[ noticing that a 2 reduces to ft, 2 under Ho 
and using the same methodology as in the proof of Theorem 13 . 11 □ 
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5. CONCLUSION 

We will now briefly summarize our constatations on simulated samples. Following 
the same methodology as in Section 5 of [1] and also being inspired by the empirical 
work of Park [31] , we have compared the empirical power of the statistical procedure 
of Theorem 14.41 with the statistical tests commonly used in time series analysis to 
detect the presence of a significant first-order correlation in the residuals. Assuming 
that 6 P ^ was a statistically significant parameter, our observations were essentially 
the same as those of [3] for different sets of parameters. Namely, on large samples 
(n = 500), we have clearly constated the asymptotic equivalence between the h-test, 
the Breusch-Godfrey test and our statistical procedure, as well as the superiority 
over the commonly used portmanteau tests. On small-sized samples (n = 30), our 
procedure has outperformed all tests by always being more sensitive to the presence 
of correlation in the residuals, except under "Ho even if the 84% of non-rejection were 
quite satisfying. Our expression of the test statistic seems therefore less vulnerable 
than the one of Durbin for small sizes. To conclude, the extension of this work to 
the stable p— order autoregressive process where the driven noise is also generated 
by a q— order autoregressive process would constitute a substantial progress in time 
series analysis. The objective would be to propose a statistical procedure to evaluate 
%o : "Pi = 0, P2 = 0, . . . , p q = 0" against the alternative hypothesis "Hi that one 
can find 1 < k < q such that pk ^ 0, based on the Durbin- Watson statistic. In 
[13], Durbin gives an outline of such a strategy which seems rather complicated 
to implement, relying on power series of infinite orders and under a Gaussianity 
assumption on the driven noise (V n ). The author strongly believes that it could 
be possible to obtain the results explicitly and under weaker assumptions, via very 
tedious calculations. A recent approach in [TO], based on saddlepoint approximations 
for ratios of quadratic forms, could form another way to tackle the problem since 
the Durbin- Watson statistic is precisely a ratio of quadratic forms. 



Appendix A 
ON SOME LINEAR ALGEBRA CALCULATIONS 



A.l. Proof of Lemma 12.11 

We start with the proof of Lemma 12.11 Our goal is to show that the matrix B 
;iven by (12.71) is invertible. Consider the decomposition B = Bi + pB 2 , where 



/ 1 



B 1 



-e 2 



— On 



\ o 



-01 

1-02 
"01 — 03 



Cp-1 

-o v 



"02 

"03 

1-04 



-o, 



'p-1 

-0„ 



—01 



Pp-l 



-01 

-02 



-0 P 





-01 











1/ 



14 



/ o 



01 



Bo 
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-1 


6 1 ... . 


01 


02 • • • • 


1+02 


03 • • • • 



Cp-2 
#p-l 



'7' 





0„ 0„ 



V P 



0p-2 



01 



P \ 









o / 



It is trivial to see that |0j + 9j\ < |0j| + \9j\ for all 1 < i,j < p, and the same 
goes for 1 — I^jI < |1 — 9i\. These inequalities immediately imply that B\ is strictly 
diagonally dominant, and thus invertible by virtue of Levy-Desplanques' theorem 
6.1.10 of [22]. Hence, B = (I p+2 +pB 2 B± 1 )Bi and the invertibility of B only depends 
on the spectral radius of pB 2 B^[ l , i.e. the supremum modulus of its eigenvalues. 
One can explicitly obtain, by a straightforward calculation, that 



B 2 B 1 



-i 



/-0i 
-1 






V o 



■1-02 



-1 



01 — 03 



0p-2 



0« 



'p-1 




-1 



P \ 








0/ 



The sum of the first row of B 2 B± 1 is —1, involving de facto that —1 is an eigenvalue 
of B 2 B^ 1 associated with the (p + 2)— dimensional eigenvector (l 1 ... l) . By 
the same way, it is clear that 1 is an eigenvalue of B 2 B^ X associated with the eigen- 
vector (1 -1 ... (-l) p+1 )'. Let P(X) = det(B 2 B^ - \I p+2 ) be the characteristic 
polynomial of B 2 B^ 1 . Then, -P(A) is recursively computable and explicitly given by 



(A.l) 



P+2 

p(a) = (-Ar 2 + j> 
fc=i 



-A) 



p+2-k 



where (bk) designates, for k 6 {1, . . . , p + 2}, the elements of the first line of B 2 B^ 1 . 
Since —1 and 1 are zeroes of -P(A), there exists a polynomial Q(X) of degree p such 
that -P(A) = (A 2 — l)Q(X), and a direct calculation shows that Q is given by 

(A.2) Q(A) = (-A) p -X>(~Ar fc . 

fc=i 

Furthermore, let -R(A) be the polynomial of degree p defined as 

v 
(A.3) J R(A) = A p -^|0 fe |A p - fe , 

fc=i 
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and note that we clearly have -R(|A|) < |Q(A)|, for all A G C Assume that Ao G C 
is an eigenvalue of B 2 B^ X such that |Ao| > 1. Then, 

p / p 

R(\M) = |Ao| p -^|4||Aor"=|A | p 1-^I^IN" 



fe=i 



k=l 



> 



|A | P 1-^1^1 >0 



k=i 



as soon as ||0||i < 1. Consequently, |Q(A )| > 0. This obviously contradicts the 
hypothesis that Ao is an eigenvalue of B 2 B^ 1 . This strategy is closely related to 
the classical result of Cauchy on the location of zeroes of algebraic polynomials, 
see e.g. Theorem 2.1 of [29]. In conclusion, all the zeroes of Q(X) lie in the unit 
circle, implying p(B 2 B^ 1 ) < 1. Since 1 and —1 are eigenvalues of B 2 B^ X , we have 
precisely p(B 2 B^ 1 ) = 1, and therefore p(pB 2 B^ 1 ) = \p\ < 1. This guarantees the 
invertibility of B under the stability conditions, achieving the proof of Lemma 12.11 
Finally, Corollary 12.11 immediately follows from Lemma 12.11 As a matter of fact, 
since B is invertible, we have det(-B) ^ 0. Denote by b the first diagonal element of 
B~ l . Since det(C) is the cofactor of the first diagonal element of B, we have 



(A.4) 



det(C) 
det(B)' 



However, it follows from ( 12. 8 p that b = X . We shall prove in the next subsection 
that the matrix A p given by ( 12. 9 p is positive definite. It clearly implies that Ao > 
which means that b > 0, so det(C) 7^ 0, and the matrix C is invertible. □ 



A.2. Proof of Lemma [US 

Let us start by proving that the spectral radius of the companion matrix associated 
with model (II. ip is strictly less than 1. By virtue of the fundamental autoregressive 
equation (IB.8P detailed in the next section, the system ( II .ip can be rewritten in the 
vectorial form, for all n > p + 1, 



(A.5) 



$r x = Oi^ti + w, 



where $^ +1 stands for the extension of ^ given by (12. ip to the next dimension, 
W n = (V n ... 0) and where the companion matrix of order p + 1 



(A.6) 



C A 



/Oi + P 

1 



V 



9 2 - OlP 

1 



dp — 6 P 





-Ip 



-0 P P\ 





J 
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Let Pa{p) = det(Cx — pJp+i) be the characteristic polynomial of Ca- Then, it follows 
from Lemma 4.1.1 of IT2] that 



p A Qi) = (-iy[n p+1 -(e 1 + p)fi p -J2(e k -e k „ 1 p)^ +l - k + e p p), 



k=2 



(A.7) = (-iY(ji-p)[iJ?-^e k if- k )=(-iY<j l -p)p<j l ) 



fc=i 



where the polynomial 



p 



/ A/V 



fc=i 



fc=i 

Assume that /i 6 C is an eigenvalue of Ca such that \po\ > 1. Then, under the 

stability condition \p\ < 1, we obviously have po ^ p- Consequently, we obtain that 

P(po) = which implies, since po ^ 0, that 

p 

(A.8) l-j>/^* = 0. 

fe=i 

Nevertheless, 

p p 

<X)|4||/^*|<X)I*I <1 

fc=l fc=l 

as soon as ||0||i < 1 which contradicts (1A.8J) . Hence, p(Ca) < 1 under the stability 
conditions ||0||i < 1 and \p\ < 1. Hereafter, let (F n ) be the stationary autoregressive 
process satisfying, for all n > p + 1, 

(A.9) ^ +1 = 04^1 + W n 

where 

^ +i = (F n F n _! ... Y n _ p y. 

It follows from (|A.9[) that, for all n > p + 1, 

p 
Y n = {9 l + p)y n _i + ^2(9k - 6fc_ 1 p)F„_ fe - OppY^p^ + K- 

fc=2 

By virtue of Theorem 4.4.2 of [9], the spectral density of the process (Y n ) is given, 
for all x in the torus T = [— ir, n], by 

(A.10) f Y (x) 



27r\A(e- ix )\ 2 
where the polynomial A is defined, for all p ^ 0, as 

(A.ll) A(p) = (-lYp^ 1 P A ^ 1 ), 

in which Pa is the polynomial given in ( 1A.7I) . and A(0) = 1. In light of foregoing, 
A has no zero on the unit circle. In addition, for all fceZ, denote by 



fk= f fy(x)e- ikx dx 
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the Fourier coefficient of order k associated with fy. It is well-known that, for all 
p > 1 , the covariance matrix of the vector \P£ coincides with the Toeplitz matrix of 



order p of the spectral density fy in ( 1A.10I) . More precisely, for all p > 1, we have 



(A.12) TO= (fi-j) =a 2 A p 

where A p is given by ( 12. 9ft and T stands for the Toeplitz operator. As a matter of 
fact, since p(Ca) < 1, we have 



lim E 

n— >oo 



n n 



E 



p p 



a 2 A p . 



Finally, we deduce from Proposition 4.5.3 of [9], or from the properties of Toeplitz 
operators deeply studied in [20], that 

(A.13) 2irm f < A min (T p (/y)) < X max (T p (f Y )) < 2nM f 

where 

rrif = min fy (x) and Mt = max fy (x) . 

x g T x g T 

Therefore, as rrif > 0, T p (fy) is positive definite, which clearly ensures that for all 
p > 1, A p is also positive definite. This achieves the proof of Lemma [2.21 □ 

Appendix B 
PROOFS OF THE AUTOREGRESSIVE PARAMETER RESULTS 

B.l. Preliminary Lemmas. 

We start with some useful technical lemmas we shall make repeatedly use of. The 
proof of Lemma [B. II may be found in the one of Corollary 1.3.21 in [T2] . 

Lemma B.l. Assume that (V n ) is a sequence of independent and identically dis- 
tributed random variables such that, for some a > 1, E[|V"i| a ] is finite. Then, 



1 n 
(B.l) lim -V |\4| a = E[|Vin a.s. 

fc=l 

and 

(B.2) sup |Vfc| = o(n 1/a ) a.s. 

l<k<n 

Lemma B.2. Assume that (V n ) is a sequence of independent and identically dis- 
tributed random variables such that, for some a > 1, E[|Vi| a ] is finite. If (X n ) 
satisfies (II. ip with \\9\\i < 1 and \p\ < 1, then 

(B.3) ^\X k \ a = 0(n) a.s. 

fc=0 

and 

(B.4) sup \X k \ = o(n 1/a ) a.s. 

0<fe<n 
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Remark B.l. In the particular case where a = 4, we obtain that 
\^X k = 0{n) a.s. and sup X\ = o(y/n) 



fc=0 



a.s. 

0<fc<n 



Proof. The reader may find an approach following essentially the same lines in the 
proof of Lemma A. 2 in [I], merely considering the stability condition \\9\\i < 1 in 
lieu of |0| < 1. □ 

Lemma B.3. Assume that the initial values Xq,Xi,...,X p -i with eo = Xq are 
square-integrable and that (V n ) is a sequence of independent and identically dis- 
tributed random variables with zero mean and variance a 2 > 0. Then, under the 
stability conditions \\9\\i < 1 and \p\ < 1, we have the almost sure convergence 

S 
(B.5) lim — = a 2 A„ a.s. 

n— >oo n 

where the matrix A p is given by (12. 9 j) . 

Proof. By adopting the same approach as the one used to prove Theorem 2.2 in [I], 
it follows from the fundamental autoregressive equation (1B.8|) . that will be detailed 
in the next section, that for all < d < p + 1, 

1 n 
lim - \~^ X k _ d V k = cr 2 5 d a.s. 

n— >oo 77, *— • 

k=l 

where d~d stands for the Kronecker delta function equal to 1 when d = 0, and 
otherwise. Denote by £ d the limiting value which verifies, by virtue of Lemma IB. 21 
together with Corollary 1.3.25 of [12], 



1 n 
lim - \~^ X k _ d X k = £ d a.s. 



n— >oo n 

fc=l 



Finally, let also L e W +2 and, for < d < p + 1, L d v e MP be vectors of limiting 
values such that, 

L={£ £i ... £ P+ i)' and L d p =(l d £ d . x ... £ d . p+1 )' . 

From (1B.8I) . an immediate development leads to 

n n n n 

/ J X k - d X k = (3 2_^ $ fc _ 1 Xfc_d — 9 p p 2_^ X k ^ p ^\X k _ d + 2_^ x k _ d v k , 
fc=l fc=l fc=l fe=l 

considering that X_i, X_2, • • • , X_ p = 0. Consequently, we obtain a set of relations 
between almost sure limits, for all < d < p -j- 1, 

(B.6) £ d = P'L d p - 1 - 9 pP £ d _ p ^ + a 2 5 d 

where £- d = £ d . Hereafter, if g? varies from to p + 1, one can build a (p + 2) x (p + 2) 
linear system of equations verifying 

(B.7) BL = a 2 e 
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where B is precisely given by (12.7ft . We know from Lemma 12.11 that under the 
stability conditions, the matrix B is invertible. Therefore, it follows that 

L = a 2 B~ x e, 

meaning via (12. 8p that L = a 2 A, or else, for all < d < p + 1, i^ — <y 2 \ d) which 
completes the proof of Lemma IB. 31 □ 



B.2. Proof of Theorem EIH 

We easily deduce from (11.11) that the process (X n ) satisfies the fundamental au- 
toregressive equation given, for all n > p + 1, by 

(B.8) X n = £'$£_! - %pX n ^. x + V n 

where /3 is given by f 12 . 5 j) . On the basis of (1B.8|) . consider the summation 

n n n n 

(B.9) E ^-xX k = E ®{-iP'K-i ~ %P E $ "-i^-p-i + E $ "-i^- 

fe=l k=\ fc=l fc=l 

First of all, an immediate calculation leads to 

(B.10) E^i^-i = (^-i-^ 

fc=i 

where S n -\ and S 1 are given in (12. 2p . Let us focus now on the more intricate term 

n 

E $ "-i x ^ 
fc=i 

in which we shall expand each element of ^_ 1 according to (IB.8[) . A direct calcu- 
lation infers the equality, for all n > p + 1, 



(B.n) e^-i^-p- 1 = 5 «-i J ^ - vE $ i^ + -iEH-iVfc+e» 
fe=i fc=i fe=i 

where Lemma [B.2l ensures that the remainder term £ n is made of isolated terms such 
that ||£ n || = o(n) a.s. Let also M n be the p— dimensional martingale 



(B.12) M n = E*£-i*V 

fc=i 

We deduce from (fB~9|) together with flB~T0|) and (iRTTj) that 

J2 ®l-i x k = aS n -i(I P - pP J p )f3 + a(/ p - 6 p pJ p )M n + a£„ 
fe=i 

where a is given by (j2.4p . Thus, taking into account the expression of the estimator 
(I2.3p . we get the main decomposition, for all n > p + 1, 

(B.13) # n = a(/ p - # p pJ p )/3 + a(S n ^Y\l p - 9 pP J p )M n + a^-i)-^. 
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For all n > 1, denote by T n the a— algebra of the events occurring up to stage 
n, T n = cr(X , . . . , X p , V\, ... , V n ). The random sequence (M n ) given by (IB.12P is 
a locally square-integrable real vector martingale p2], [21], adapted to J-" n , with 
predictable quadratic variation given, for all n > 1, by 



(M) n = ^E[(AAf fe )(AAf,) / |^-i], 



fc=i 



(B.14) = a 2 Y,$i-M'-i = ° 2 {Sn-i-S) 



fc=i 



where AM^ stands for the difference M\. — M^_\. We know from Lemma [B. 31 that 
(B.15) lim — = a 2 A„ a.s. 

and A p is positive definite as a result of Lemma 12.21 Then, (IB.15P implies that 
(B.16) lim t -^A = a 2 p \ Q a . s . 

n— >oo 77, 

where Ao > 0. Moreover, since A p is positive definite, we also have that 

(B.17) A max (S'„) = O (XrmniSn)) a.S. 

Consequently, we deduce from (IB. 141) . (IB. 161) . (IB. 171) and the strong law of large 
numbers for vector martingales given e.g. in Theorem 4.3.15 of [12] . or [13] that, 

(B.18) lim(M); 1 M n = a.s. 

and obviously, 

(B.19) lim(S n ^ 1 y 1 (I p -9 p pJ p )M n = a.s. 

n— >oo 

As mentioned above, (V n ) having a finite moment of order 2 implies, via Lemma 
EJand flB~T5|) . that 



(B.20) lim (S'n-i) -1 ^ = a.s. 

n— >oo 

Finally, (1B.13J) together with ( TB.19J) and ( ]B.20p achieve the proof of Theorem I2.1[ 

lim 6 n = a(I p — p pJ p )f3 a.s. 



□ 



B.3. Proof of Theorem EIS 

The main decomposition (IB. 13|) enables us to write, for all n > p -+- 1, 

(B.2i) ^ (e n - r) = oV^n-i)-^ - e pP j p )M n + av^s^r 1 ^. 

On the one hand, we have from Lemma lB.21 with a = 4 that ||£ n || = o(^Jn) a.s. 
assuming the existence of a finite moment of order 4 for (V n ). Hence, via (1B.15J) . 

(B.22) lim V^ (S n ^y l £ n = a.s. 



V 
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On the other hand, we shall make use of the central limit theorem for vector mar- 
tingales given e.g. by Corollary 2.1.10 of [12], to establish the asymptotic normality 
of the first term in the right-hand side of ( IB. 211) . Foremost, it is necessary to prove 
that the Lindeberg's condition is satisfied. We have to prove that, for all e > 0, 

1 n 

(B- 23 ) -E E [H AM ^I| 2I {||AA4||>^}I^- 

fc=i 

where AM k = M k - M fc _i = ^l^Vj,. We have from Lemma [R2] with a = 4 that 

(B.24) £ll*E-il| 4 = 0(n) a - s - 

it=i 
Moreover, for all e > 0, 

_. n ^ n 

-E E [n AM ^i 2l {iiAM fcl i>^}i^-i] < ^E E 0i A ^ii 4 i^- 1 ], 

fc=l k=l 

4 n 

< _^Y\\<s>i i|4 

k=l 

where r 4 stands for the moment of order 4 associated with (V n ). Consequently, 
( 1B.24J) ensures that 

-J2 E [\\ AM ^ 2l {\\AM k \\>sVE}\^-i] =0(n- 1 ) a.s. 
fc=i 

and the Lindeberg's condition (1B.23|) is satisfied. We conclude from the central limit 
theorem for vector martingales together with Lemma 12.21 and Lemma IB. 31 that 

(B.25) yfH {M)~ 1 M n A M (0, a^A; 1 ) 

where A p is given by (I2.9|) . which leads to 

(B.26) a,/E (Sr^y 1 (I p - 9 pP J p ) M n A M (0, E ) . 

Finally, (TB~2ll . dB~22l) and ( TB~26l) complete the proof of Theorem E2 □ 



B.4. Proof of Theorem [HE 

Let (W n ) be the sequence of standardization matrices defined as W n = \fnl p . 
Consider the locally square-integrable real vector martingale (M n ) with predictable 
quadratic variation (M) n given by (IB.14[) . Via Lemma [B.3| we have the almost sure 



convergence 

(B.27) lim W- 1 (M) n W~ x = a 4 A p a.s. 

n—too 

where A p is given by (12. 9p . For all n > 0, denote 



:B.28) T n = Y,X, 



i 

k 

k=l 
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with T n = 0. From Lemma [B.2I with a = 4, we have that T„ = O(n) a.s. Thus, 



E X n ^-^ ^n ~ Tn-1 __ ST^ ( 2n + 1 

n 2 *-~i n 2 2-^i \ n 2 ( n + \y 

n=l n=l n=l x v ' 



T 



,n=l 



n- 



,n=l 



■/?- 



which immediately implies that 
(B.29) 



< +oo a.s. 



Ild> p II 4 

n < +oo a.s. 



n=X 



n- 



From (IB. 271) and (1B.29[) . we can deduce that (M n ) satisfies the quadratic strong law 
for vector martingales given e.g. by Theorem 2.1 of |TTj . 



1 
lim > 

n->oo log 7l p ^— ' 



fc P 



Wfc-^fcMfc' W^ 1 = a 4 A p a.s. 






(B.30) 

v 7 n-^oologriP^L {k + l) p 

Hereafter, it follows from (IB.13P that, for all n > p + 1 
n - 0*j (9 n - 9*j = a 2 (S n ^)- 1 

(B.31) = a 2 (S n . 1 y 1 K p M n M;K p (S n . 1 

where K v = (I p — 9 p pJ p ) and the remainder term 

C„ = a^S^y^CnM^Kp + K p M n i' n + ^QiSn-iT 1 . 
However, we have from Lemma [2.21 and Lemma [B.3I that 



T3.32) 



lim n(S n _i 



v-l 



-2A-1 



a~ 2 A 



a.s. 



As a result, (IB.30J) . (IB.32P and a set of additional steps of calculation lead to the 
almost sure convergence 



(B.33) 



n 

lim V^-ir^MfeM^CSfc-i)- 1 =i^A-% 

ra->oc logn ^ 



a.s. 



fc=i 



since i^, A"" 1 = A" 1 ^ due to the bisymmetry of A" 1 . Assuming a finite moment 
of order 4 for (V n ), one can easily be convinced that £ n is going to play a negligible 
role compared to the first one in the right-hand side of (IB.3ip . Indeed, we clearly 
have that ||M n ||||£ n || = o(n 3 ^y/\ogn) a.s. It follows that 



(B.34) 






0(1) a.s. 



Finally, ( 1B.33I) and ( IB. 341) complete the proof of the first part of Theorem 12.31 



i im — J2(e k - e*) (e k - 9*) = s e 

n->oo log n *■ — ' \ / \ / 

& fc=i 



a.s. 



since Eg = a 2 K p A p x K p . 
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The law of iterated logarithm ( 12.131) is much more easy to handle. It is based on 
the law of iterated logarithm for vector martingales given e.g. by Lemma C.2 in [TJ. 
Under the assumption ( IB. 291) already verified, for any vector v G MP, we have 

/ n V /2 ( n \ 1/2 

limsup — - v\S n ^y l M n = -hminf —- ■ v'{S n ^y l M n , 

n^oo \21oglogn/ n^oo \. 2 log log n / 



(B.35) = Jv'A-iy a.s. 

Via (IB.35P and the negligibility of £ n , we immediately get 



1/2 / \l/2 



limsup I — — J v'(9 n -9*) = -liminfl — — ) v' ( 

n ^oo \21oglogny V / n^co \2Ioglogny V 

(B.36) = aJv , K p A- 1 K p v a.s. 



Since (1B.36J) is true whatever the value of v G W, we obtain a matrix formulation 



of the law of iterated logarithm, 



n \ /^ \ /^ \ / 



limSUp( 21oglogJ ( ^-nK- r = ^ a.s. 



Passing through the trace in (IB. 371) . we find that 



/ Tl \ 1 1 ^ 1 1 9 

(B.38) limsup — — : e B -6* =tr(S e ) a.s. 

which completes the proof of Theorem 12.31 n 



Appendix C 
PROOFS OF THE SERIAL CORRELATION PARAMETER RESULTS 

C.l. Proof of Theorem ED 

Let us introduce some additional notations to make this technical proof more 
understandable. Recall that, for all d G {0, . . . , p + 1}, we have the almost sure 
convergence 

1 n 
(C.l) lim ~y^X k _ d X k = a 2 \ d a.s. 

n-»oo n *-~* 
fc=l 

Let A° , Ap and Al be a set of p— dimensional vectors of limiting values such that, 
for d= {0,1,2}, 

(C.2) A p = (A<2 Xd+i ■ ■ ■ Xd+p-i) , 

and note that the almost sure convergence follows, 

1 ™ 
(C.3) lim - J2 & k - d X k = * 2 A d p a.s. 

k=\ 
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For all n > 1, denote by A n the square matrix of order p defined as 

n 

fe=i 



Following a reasoning very similar to the proof of Theorem 12. 1[ it is possible to 
obtain the decomposition, for all n > p + 1, 

n n n 

(c.5) E $£x fe = i n r + ff ^^y,- a e pP j p J2 $*_ 2 ^ + vn 
fc=i fc=i ^=1 

where the residual 7/ n is made of isolated terms such that \\rj n \\ = o{n) a.s. As an 
immediate consequence, we have the relation between the limiting values 



(C.6) 



A° = A v e* + 



ae 



where the almost sure limiting matrix of a 2 A n /n is given by 

/ Ai A2 A3 A p ^ 



(C.7) 



A, 



An 



Ai Aj 



A 



\Ap_2 A p _3 A p _4 



p-1 



Ai/ 



The reader may find more details about the way to establish these almost sure 
convergences e.g. in the proof of Lemma [B.31 Likewise, one proves that 

(C.8) 



A^ = A' 6* — a 9 p pJp e. 



Finally, the very definition of the estimator 9 n directly implies another relation, 
involving the matrix A p given by (12.91) . 

(C.9) Aj = Ap0*. 

Relations ( IC.6J) . (1C.8J) and ( 1C.9J) will be useful thereafter. Let us now consider the 
expression of p n given by (I3.2p . On the one hand, in light of foregoing, 

1 n 1 n 

n— >oo Ti ^- -" ' n— >oo 7| *— ' \ / \ / 

k=l 

a 2 (\ l -(A p , + A 2 p , ^e* + e* , A p e^ 

^(Xx-Al'O* -a9{ 



k=l 



a.s. 



(CIO) 

On the other hand, similarly, 

1 n 1 n 

n— >oo 77, ^— — ' n->oo n *— ' \ / 



A=l 



fc=l 

a 2 fA -2Ayr + 6*'A P 0* 



(Cll) 



a 2 (A -A!> 



a.s. 
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Via the set of relations ( IB. 61) . we find that Ao = j3 'A* — 9 p p\ p+ i + 1 for d = 0, and 
\ p+ i = (3'J p Ap — 9 p pX for d — p + 1, in particular. Hence, with 9* = a(I p — 9 p pJ p )(3, 

\ 1 -A 2 p '9*-a9* 1 = Ai-Aj'0* - a91(\ - f3'A l p + 9 p p\ p+1 ), 

= Ai - Aj V - c^(A - /9'AJ + M/3'J p Aj - ^pAo)), 

= Ai-AJV-^Ao-AJV), 

(C.12) = A 1 -A2 / r-(0 1 + p)(A o -Ai / r) + ^;(Ao-A; / 

since one has to note that 9\ = 9 X + p - 9 p p9*. Via (ICl)]) . Ai = A° p '9*. Thus 
Ai-A?> = ^'(A°-A 2 p 



p \ p p n 

9*'A' V 9* - 9*'A P 9* + a(9i + p), 
a(0i + p)(A - /3'Ai + 9 p pX p+1 ), 



V 

9 

p 



(C.13) = (0i + p)(A o -Aj'0*). 

To conclude, ( 1C.12I1 together with ( 1C.13J) lead to 

Al _ aj> _ a oi = e pP 9;(\ - Aj V) 

which, via flC.lQj) and (jC.ll[) . achieves the proof of Theorem 13.11 

lim p n = 9 p p9* a.s. 



□ 



C.2. Proof of Theorem S21 



First of all, we have already seen from ( IB. 131) that, for all n > p + 1 



(C.14) 5 n _! (e n - r) = a(I p - 9 pP J p )M n + a£ n 

where Lemma lB.21 involves ||£ n || = o{y/n) a.s., assuming a finite moment of order 
4 for (V n ). Our goal is to find a similar decomposition for p n — p*. For a better 
readability, let us introduce two specific notations Y n and Z n given by 

Y n = X n — p X n _i and Z n = X n _i — p X n . 

We also note Y£ = {Y n F n _i . . . Y n ^ p+1 ) and Z v n = (Z n Z n _i . . . Z n „ p+ i) . 
Denote by F n the recurrent p— dimensional expression that appears repeatedly in 
the decomposition, given, for all n > 1, by 

(C15) F n = & n P'Z* - [zi_ x + Y*) x n . 

From the residual estimation (13. ip . the development of p n — p* reduces to 

(c.i6) J n ~i(pn - p*) = w n + (e n - e*y ' H n 
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where H n is a p— dimensional vector and, for all n > p + 1, 



(C.17) 
(C.18) 
(C.19) 



J, 



= Eft 2 . 

fc=0 

n n 

fc=l fc=l 

n n 

# n = EC^'^ + ^ + E** (&-**) zj + a^, 



ir, 



fc=i 



fc=i 



with ||/i n || = o(y/n) a.s. and z/ n = o(y/n) a.s. The reasoning develops in two stages. 
At first, we shall prove that W n reduces to a martingale, except for a residual term. 
Then, using Theorem 12.21 and the central limit theorem for vector martingales, we 
will be in the position to prove the joint asymptotic normality of our estimates. 

Let C be the square submatrix of order p + 1 obtained by removing from B given 
by (12. 7p its first row and first column, 

/ 1-ft -ft .... 

-ft -ft 1-ft ••• • 



(C.20) 



C 



-Pp 

o pP 







V 



-fip-i + Opp —ft- 2 
Pp Pp— i 



-Pi 







-ft -ft 1/ 



By Corollary 12.11 we have already seen that the matrix C is invertible under the 
stability conditions. Denote by N n be the (p + 1)— dimensional martingale 



(C.21) 



tf» = E*£to 



^=1 



where $^ +1 stands for the extension of $^ to the next dimension. A straightforward 
calculation based on (1B.8J) shows that the following linear system is satisfied, 

n n 

C^t\X k = TY,X 2 k + N n 



k=i 



fc=i 



in which T is defined as 

(c.22) r=(ft ft ... ft -M'- 

As a result of the invertibility of C, we get the substantial equality, for all n > p-\- 1, 



(C.23) 



E *E** = c~ lT E x * + ^^ 



/c=i 



fc=i 



A large manipulation of W n given in (1C.18I) still based on the fundamental autore- 
gressive form (IB. 81) shows, after further calculations, that there exists an isolated 
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term v n such that v n = o{y/n) a.s., and, for all n > p + 1 . 

n n n 



fe=l fc=l jfc=l 



+ a P *e*'(i p - e pP j p ) J2 <%-& + v n , 

fe=i 

leading, together with ( 1C.23I) . to 

n 

(C.24) W n = (G'C- 1 T - p* - a0l)J2 x2 k + G'C~ l N n + L n + v n 

fe=i 

where, for all n > p + 1, 

fc=i / fc=i 

and where the (p + 1)— dimensional vector G is given by 

(C.26) G = p*ti* + a9lT-S* 

with??* = (0J 9* ... 6; 0)' and <5* = (-1 0? ... 9*^ 9;)'. In terms of 
almost sure limits, by using the same methodology as e.g. in the proof of Lemma 
IR31 flCl23|) directly implies 

(C.27) A l7- 1 T = AJ +1 

where A p+1 = (Ai A2 ... A p +i) is the extension of A* in (1C2|) to the next di- 
mension. Hence, following the same lines as in the proof of Theorem 13. 1[ 

\ (G'C- 1 T-p*-a9* 1 ) = G f A p+l -X {p* + a9* 1 ), 

= p*(Al'9* - Ac) + a9* 1 (T'A 1 p+1 - A ) + (A x - A 2 p '9*), 

= 9* 1 (aA 1 p '(I p - 9 pP J p )f3 - a{\ - 9 p p)(l + 9 p p)X ) 

+p*(A p / r-A ) + (A 1 -Aj , r), 

= 9l(Al'9* - A ) + p*(Al'9* - A ) + (Ai - A 2 p 9*), 
= -a(p* + 9\) + a{p* + 9\) = 0. 

One can see from Lemma [2.21 that Ao > 0. The latter development ensures that the 
pathological term of (1C.24J) vanishes, as it should. Finally, W n reduces to 

(C.28) W n = G'C~ 1 N n + L n + u tu 

and one shall observe that G'C~ 1 N n + L n is a locally square-integrable real mar- 
tingale [12], EU- One is now able to combine (IUT4J) and (1UT6]l via (1TI28D . to 
establish the decomposition, for all n > p + 1, 

(C.29) J n _x (p n - p*) = G'lT 1 ]^ + L n + <(/„ - ^pJ p )( 1 S„_ i r 1 ^ n + r n 



28 FREDERIC PROIA 

where the remainder term r n = a^ t ' n {S n -i)~ 1 H n + v n is such that r n = o{^/n) a.s. 
Taking tediously advantage of the (p + 2) x {p + 2) linear system of equations (1B.6J) . 
one shall observe that G'C^ 1 = a \U' it p +i) with 

U p ={l + (3 2 P3-P1 ••• Pp-Pp-2 -Pp-i-epp)', 
and u p+ i = —a~ 1 0* — Q p pQ\. The combination of (IC.25I) and (1C.28J) results in 

n 

(C.30) W n = a (U p + (I p - 9 pP J p )(p*9* - r*))' M n - 6* p ^ X k _ p . x V k + u n 

k=i 

where r* — (9^ #3 • • • #p 0) . Consequently, it follows from fl0.14D together with 
(I029|) and (IC3U|) that 

(C.31) v^ (i n ~ 6 l) = A=P n N n + R n 

\Pn- P J s/n 



where the square matrix P n of order p + 1 is given by 

(pt x) 

( C - 32 ) P n = L(2,l) pp.2) 

with 

^ 2,1) = M^-i) -1 (« (t/ p + (i p - e pP j p )(p*e* - r*))' + aHKs^y 1 ^ - e pP j p )) , 

^i 2 ' 2) = -"(Jn-xr 1 ^ 

and where the (p + 1)— dimensional remainder term 

(C.33) ^^(tl"-^) 

is such that ||-R n || = o(l) a.s. Via some simplifications on H n , ( IC.6J) . (1C8|) and 
(IC.9J) . we obtain that 

TJ 

(C.34) lim — - = — a{I p — 6 p p.J p )e a.s. 

Furthermore, it is not hard to see, via Lemma [B. 3 1 ( IC.llj) . (1C34|) and some simpli- 
fications on Pn , that 

(C.35) lim P n = a- 2 P a.s. 

n— >oo 

where P is the limiting matrix precisely given by (13 .4p . The locally square- integrable 
real vector martingale (N n ) introduced in (1C.21J) and adapted to T n has a predictable 
quadratic variation (N) n such that 

(C.36) lim ^^ = a 4 A„ +1 a.s. 

where A p+ i is given by ( 13. 51) . This convergence can be achieved following e.g. the 
same lines as in the proof of Lemma IB. 31 On top of that, we also immediately 
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deduce from ( IB. 241) that (N n ) satisfies the Lindeberg's condition. We conclude from 



the central limit theorem for martingales, given e.g. in Corollary 2.1.10 of [12], that 

(C.37) -^N n -^Af(0,a 4 A p+1 ). 

Whence, from (1031"]) . fl033|) . (ICl35|) . ( I037J) and Slutsky's lemma, 



(C.38) v^ f ? n _ 9 l) A A/" (0, PA P+1 P') . 

This concludes the proof of Theorem 13 .21 where, for readability purposes, we omitted 
most of the calculations which the attentive reader might easily deduce. □ 



C.3. Proof of Theorem S3 



In the proof of Theorem l3.2l we have established a particular relation that we shall 
develop from now on, to achieve the proof of Theorem 13.31 Indeed, from (1C.31I) . for 
all n > p + 1 , 

(C.39) Pn~P* = n~\' n N n + (Jn-i)- 1 r n 

where N n and J n -\ are given by (IC.21I) and (IC.17p . respectively, where r n is such 
that r n = o(y/n) a.s. and where 7r n of order p + 1 is given from ( 1C.32I) by 

(C.40) Kn=(P^ l) P™)'. 

Denote by ix the almost sure limit of 7r n , accordingly given by 

(C.41) 7r = a~ 2 (P> L <p)' 

where Pl and (p are defined in ( 13. 4ft . Hence, ( 1C.39I) can be rewritten as 

(C.42) Pn-P* = n- l ix'N n + n' 1 (vr n - vr)' N n + ( J^i)" 1 r„. 

One can note that (ir'N n ) is a locally square-integrable real martingale with pre- 
dictable quadratic variation given, for all n > 1, by 

(C.43) (VJV) n = crV(T n _ 1 -T)7r 

where the square matrix T n of order p + 1 is the extension of 5* n given by ( 12. 2 j) to 
the next dimension defined, for all n > 1, as 

(C.44) T n = J]^ +1 <l>f +1, + T, 

fc=i 

and T is a symmetric positive definite matrix. In addition, (ir'N n ) satisfies a non- 
explosion condition summarized by 

lim ^— ^ = a.s. 

n-S>oo TC Tn 71 
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by application of Lemma IB. 21 with a = 4. By virtue of the quadratic strong law for 
martingales given e.g. by Theorem 3 of [2] or [3J, 

1 ^f n'N k 



(C.45) lim > — = — a.s. 

n^oo logn ^ \-K'T k _i1Xj Tl' A p+1 71 

where A p+1 given by (13. 5p is the almost sure limit of a~ 2 T n /n. We refer the reader 
to Lemma IB.3I to have more details on the latter remark. Note that it' A p+ i it > 
since A p+1 is a positive definite matrix, as a result of Lemma 12.21 The same goes 
for it' T n 7T, for all n > 1, assuming a suitable choice of T. Besides, the almost sure 
convergence of ir n to it, the finite moment of order 4 for (V n ) together with (IC.45P 
ensure that 

(n k -it)'N k r k \ 2 ^ ( s^ (i^k ~ n)' N k )' 



fc=i - Jk ~^ U=i 



E(^^ + tM =o£^^p*_ + E 




0(l) + o E 



(TT'iVfc) 2 



k 2 
k=l 



(C.46) = o(logn) a.s. 

since r n is made of isolated terms of order 2 and J n = 0(n) a.s. It follows that 



lim > \Pk — P*) = lim -, / ( — ; — 

rwoo log 71 z — ' V / rwoo log Tl ^-^ \ K J 



2 



1 ^V *'N k \ 2 f^T k -m x 

lim 



n->oo logn ^ \7l'T k _i TT J \ k 

qyA p+l7 r) 2 4 

= — = a % A p+ i ix a.s. 

IT 1 A p+ i TT 

via ( IC.45P and (1C46[) . since the cross-term also plays a negligible role compared to 
the leading one. The definition of tt in ( IC.4ip combined with the one of T in (13.61) 
achieves the proof of the first part of Theorem 13.31 

Furthermore, it follows from the law of iterated logarithm for martingales [33] 
see also Corollary 6.4.25 of [12], that 



(n'N) n V /2 rr'N n f (n'N) n \ 1/2 tt'N, 

lim sup — — - — - — — — - — — 7— = — lim ml 



21oglog(7r'A0 n ; (7T'N) n n~>oo \2\og\og(7r'N) n J (tt'N), 

1 a.s. 



since we have via (IB.29[) that 



(C47) J2 



U'<f>P ^ 



k 

k=i 



k-iJ . . 

- < +oo a.s. 



Recall that we have the almost sure convergence 

(C.48) lim ^' N ^ n = crVA p+ i7r a.s. 
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Therefore, we immediately obtain that 

1/2 n'N n 



lim sup 



n 



n 



liminf . 

ra->oc \2 log log nj [TC'Njj 



1/2 7T>N n 



a~ 2 (n'A p+1 ny 1 / 2 



a.s. 



, 2 log log n/ (n'N) n 

(C.49) u K n ^ p+1 

As in the previous proof and by virtue of the same arguments, one can easily be 
convinced that the remainder term in the right-hand side of (1C.42I) is negligible. It 
follows from ([042]) together with (IQ481) and ([049]) that, 

1/2 / \ 1/2 



lim sup 



n 



2 log log n 



Pn~ P 



liminf ! 

ra-kx) \ 2 log log n 



Pn- P 



a \/n'/\ p+ in a.s. 



which achieves the proof of Theorem [3] 



□ 



Appendix D 

COMPARISON WITH THE H-TEST OF DURBIN 

We shall now compare our statistical procedure with the well-known h-test of 
Durbin [14] . Let us assume that Tio is true, that is p = 0. Then, the least squares 
estimate of the variance of 6 n is given by 

(D.l) ¥ n (6 n ) = a^S-l 1 

where S n is given in (12.21) and a 2 is the strongly consistent least squares estimate 
of a 2 under Ho, defined as 



(D.2) 



o: 



i n 



fc=0 



For this proof, we use a Toeplitz version of i9 n given by 



o 



op 



( 4 

s, 

s 



w 



p-1 



}P- 2 

II 



>p— 3 



P-1n 
n 
,P-2 

n 

;P-3 



s«) 



where, for all < h < p, s% = Xlfe=o ^kXu-h, and we easily note that S£ = S n + o(n) 
a.s. We assume for the sake of simplicity that S% is invertible, saving us from adding 
a positive definite matrix S without loss of generality. We also define 



n* 



s 1 s 2 



and d*r x = (d 1>n d 2>n 



r) 



p—l, n 



with Il n = IIP, ir n = LT^ 1 and i?„ = (S£) 1 H n is the Yule- Walker estimator. First, 
a simple calculation from ( ID.2P shows that 



(D.3) 



n°n =Sn-K^n 
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where a% is built from d n . In addition, the first diagonal element of (Sfi) -1 is the 
inverse of the Schur complement of S^ 1 in S%, given by 

The conjunction of ( ID. 31) and ( 1D.4I) leads to 

(D.5) l-nV n (£ N an_/3n 



l,nJ 



with 

a n = 5°-<(ST 1 r 1 7r« and & = S ° - 11^ ( S^)' 1 U n . 

We also easily establish, via some straightforward calculations, that 

vr n = fc n (/ p _x + d p>n J p _i) ST 1 C" 1 with fc„ = (l - d p 2 n y 
leading, since S%~ 1 is bissymetric and commutes with J p -%, to 

ru - q° — h it 1 7?P _1 — h i? it' T , S p ~ l nnH tt' f , i9 p_1 - c? _ ,? q ( 
ct n o n n. n /i n (/ n rv n Up n n n Op— i u n anu. ii n Op—\ u n o n <Jp t n°r 

Hence, from the previous results, 

h~ l rv h~ l ( s° — h -rr' i9 p_1 — h i? s p -I- h i? 2 «s ° I 



'^i ' l ^i L'm o r)_ n. ^ 



P 
P,n J m 



(D.6) = s n °-n^ n = /3 n . 

We now easily conclude from (ID.5J) and (ID.6J) that 



l-nV n (#i, n ) = # 



2 



Considering now that (S£) l = S^ 1 + o(n *) a.s. and making use of 9 n given by 
(I2.3p . it is straightforward to obtain that 6 n = $ n + o(l) a.s. and that 



2 

which ends the proof. D 



l-nY n (6 ltn )=6 p 2 n + o(l) a.s. 
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